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Problem5  In  Servomechanism  Synthesis 


Report  No*  2 


by!  Richard  C.  Lyman 

Project  Engineer  in  Electrical  Engineering 


Progress 


This  is  the  second  of  a series  of  reports  of  an  investigation  of  servo- 
mechanism synthesis  xmder  Navy  Contracts  N7ori30306  and  N7ori30308.  The 
investigation  was  initiated  originally  for  the  purpose  of  studying  the  appJ.i- 
cation  of  magnetic  amplifiers  in  servomschanism  design*  It  rapidly  became 
apparent,  however,  that  (A)  it  has  already  been  demonstrated  that  magnetic 
amplifiers  are  applicable  to  servomechanism  problems  in  a wide  variety  of  cases, 

(B)  that  such  applications  are  commonly  made  on  a rule-of-thumb  basis  and  that, 

(C)  a more  refined  application  of  magnetic  amplifiers  in  servomechanism  design 
cannot  be  separated  from  the  general  problems  of  servomechanism  synthesis  and 
that  this  project  should  start  with  the  redefinition  of  the  basic  problem,  that 
of  an  optimum  criterion  for  synthesis  on  a realistic  basis* 

In  the  classical  approach  to  servomechanism  analysis  it  is  assumed 
that  (l)  signals  are  of  regular  functional  form,  typically  step  functions, 

(2)  such  artificial  criteria  as  phase  margin  or  peak  overshoot  values  are 
satisfactory  as  measures  of  synthesis  success.  More  recently  servomechanism 
approaches  have  been  directed  to  consider  (l)  the  statistical  properties  of 
signals,  (2)  realistic  criteria  both  linear  and  nonlinear,  and  (3)  the  treatiwnt 
of  systems  which  in  themselves  are  digital  in  nature,  their  utilization  being 
effected  by  quantizing  the  analogue  signal  with  a transducer  when  necessary* 

This  is  the  direction  undertaken  in  this  particular  study. 

The  results  of  this  study  to  date  are  as  follows: 
a)  A method  of  evaluating  a linear  servomechanism  system  in  the  light  of 
any  type  of  signals  that  can  be  statistically  described,  and  any  type  of  error 
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criteria  for  which  u power  aeries  aporoximation  can  be  written  (with  the  common 
quadratic  norm  as  one  term  of  the  series).  This  work  was  described  as  report  n 
number  one. 

b)  A method  of  prescribing  the  optimum  transfer  function  for  systems 
barring  cex*tain  specified  time-varying  characteristics,  or  for  signals  having 
certain  specified  time-varying  properties  (including  random  variations),  as  well 
as  the  best  adjustment  for  first  and  second  order  systems  for  a set  of  commonly 
assumed,  statistically  described  signals  which  also  may  have  time  variations. 

In  some  respects  this  work  is  an  extension  of  Wiener's  methods.  This  work  is 
described  herein  as  report  number  two  of  the  series,  and  was  submitted  to 
Carnegie  Institute  of  Technology  by  R.  C.  Lyman  as  a Dissertation  in  partial 
fulfillment  of  the  requiremants  for  the  degree  of  Doctor  of  Philosophy  in 
Electrical  Engineering. 

c)  A method  of  deriving  a transfer  function  for  a magnetic  amplifier 
starting  with  the  B-H  characteristics  of  the  non-linear  material.  The  method 
permits  determining  in  addition  to  the  linear  terms  the  harmonic  distortion  terms 
of  both  the  envelope  and  the  carrier  functions.  This  material  is  being  readied 
for  report  in  the  near  future. 

d)  A new  mathematical-graphical  method  of  solving  problems  in  combinational 
switching  circuits.  The  importance  cannot  be  overemphasized  of  the  use  of  magnetic 
amplifiers  as  components  of  digital  circuits,  in  which  their  inherent  non- 
linearities  are  utilized  for  pulse  reshaping  rather  than  being  a severe  liability. 
Such  use  can  proceed  only  at  a limited  pace  until  the  mathematics  of  the  design 

of  digital  circuitry  are  better  known  and  in  convenient  form.  The  new  method  is 
essentially  ready  for  report. 

l)  Symposiim  ou  Statistical  Methods  in  Communication  Engineering,  Berkeley 
Campus,  University  of  California,  Aug,  17*“ IS,  1953,  ’’Ceneralized  Servomechanism 
Evaluation",  Caywood,  Lyman,  and  KaufrAan, 
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2)  I.R.E,  1954  National  Convention,  "Optimization  of  Servuayatenis'*,  Lyman  and 


Csywood,  Session:  Radio  Telemetry  and  Remote  Control  III. 
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SYNOPSIS 


This  dlsBortation  derives  optimum  transfer  functions 
for  servos ys terns.  For  the  ease  of  time-varying  signal  and 
noise  input  spectra,  the  optimum  system  has  zero  phase 
shift  and  an  attenuation  given  in  terms  of  the  input  spectra 
and  their  variances.  For  the  case  of  a time-varying  system 
parameter,  a similar  oonoluslon  results.  For  a physically 
realizable  system  of  a specified  order,  a formulation  of  the 
general  solution  is  given  and  is  solved  for  six  important 
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IHTRODaCTION 

The  Problem 

This  dlsaertatlon  oonalders  the  problem  of  the  synthesis 
of  optimum  servosystems  which  are  subjected  to  signal  Inputs 
contaminated  with  noise.  Of  primary  Interest  are  those 
problems  wuloh  are  characterized  by  time-variations  In  the 
system  parameters  and  In  the  statistical  properties  of  the 
signal  and  noise  Inputs,  The  criterion  of  system  optimization 
used  In  this  dissertation  is  the  minimization  of  the  mean 
square  error  In  the  response  of  the  servosyatem. 

The  method  of  solution  presented  herein  Is  primarily 
carried  out  in  the  frequency-plane  rathtr  than  in  the  tlme- 
plane.  Hence  the  concept  of  the  frequency  components 
present  In  the  system  Inputs,,  output,  and  error,  as  deoorlbed 
by  their  spectral  densities,  is  used  rather  than  the  concept 
of  '^helr  functional  form  with  respect  to  time.  The  method 
Is  limited  to  consideration  of  linear  systems. 

The  literature  contains  a great  amount  of  work  (lb, 2) 
which  deals  with  the  problem  of  the  optimization  of  invariant 
systems  subjeoted  to  Inputs  having  invariant  statistical 
properties.  However,,  the  optimization  problem  which  is 
characterized  by  time-variations  in  the  pareuaeters  of  the 
system  and  in  the  statistical  properties  of  the  input  has 
not  been  treated  in  the  literature.  This  dissertation 
constitutes  an  original  and  useful  contribution  to  the  field 
of  eerveaystem  synthesis  and  analysis  by  formulatlrig  the 
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latter  problem  and  presenting  solutions  of  it. 

Choice  of  Criterion 

There  are  several  reasons  for  the  choice  of  minimum 
mean  square  error  as  a criterion  of  system  optimization. 

The  principal  one  la  the  possibility  of  transformation  of  the 
mean  square  error  of  a system  response  from  the  time-plane 
directly  Into  the  frequency-plane.  That  this  Is  true  has  been 
shown  many  times  In  the  literature  (1).  In  addition,  this 
property  of  direct  equivalence  Is  valid  only  for  the  mean 
square  criterion,  not  for  any  other  powers.  Direct 
transformation  Is  desirable  In  that  system  responses  euid 
criteria  are  most  naturally  Interpreted  In  the  tlme-jiane, 
whereas  mathematical  analysis  Is  generally  handled  more 
conveniently  in  the  frequency-plane.  That  the  mean  square 
value  of  the  error  In  the  time-plane  can  bo  computed  as  the 
Integral  of  error  spectral  density  over  all  real,  positive 
frequencies  has  been  proven  In  the  literature  (la).  Another 
reason  for  this  choice  lies  In  Its  suitability  for  a class 
of  problems  Including  those  which  require  minimization  of  the 
power  losses  resulting  from  the  error  In  system  response. 

For  the  case  of  a second  order  syetem  subjected  to  step 
function  signals,  it  Is  Interesting  to  note  the  following 
result.  Minimization  of  the  average  error  power  (mean 
square  error)  specifies  a ratio  of  damping  to  critical  of 
0.5.  which  is  a relative  first  overshoot  of  O.I64.  Such  a 
response  is  generally  considered  to  be  too  oscillatory, 
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e.g.  a region  of  0.6  to  1.0  has  recently  been  advised  (2b). 

The  Statistical  Approach 

An  important  oharaoteristio  of  the  method  of  solution 
presented  herein  is  the  use  of  statistical  properties  rather 
than  functions  of  time  to  describe  the  system  inputs,  output, 
and  error.  A system  subjected  to  an  input  which  is  known  as 
a function  of  time  is  merely  a signal  generator.  For  the 
input  to  contain  any  useful  information  for  controlling  the 
system,  it  cannot  be  known  with  certainty  at  all  future  in- 
stants of  time.  It  may  be  described  only  by  its  statistical 
properties.  These  properties,  which  may  be  spectral  densities 
or  correlation  functions,  must  be  obtained  by  certain  averag- 
ing operatioQs  performed  on  the  inputs  expected  over  some 
period  of  operation  of  the  system  being  designed.  Such  sta- 
tistical Information  can  only  lead  to  the  statistical  prop- 
erties of  the  system  error,  such  as  the  error  spectral  density 
which  is  used  in  this  method.  As  a result  of  this  statistical 
approach  to  the  optimization  problem,  an  average  of  the  sys- 
tem performance  is  optimized  rather  than  any  parcioular  char- 
actoristics  of  the  system's  response  as  a function  of  time. 

The  optimum  system  may  be  considered  as  that  system  which 
achieves  the  best  compromise  between  passing  the  signal 
spectrum  (and  necessarily  some  noise  along  with  it)  and 
attenuating  the  noise  spectrum  (and  some  signal  al'  ng  with 
it) . 
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Tlme-ygrylnp;  Spectra 

The  term  "time -varying  signal  spectrum",  as  used  in 
this  dissertation,  is  a description  of  a signal  having  two 
particular  oharaoteristioa.  First,  its  statistical  proper- 
ties remain  time -invariant  over  periods  of  time  of  a suffi- 
cient length  to  properly  establish  those  properties.  They 
may  be  established  by  measurement  of  its  spectral  density 
or  its  autocorrelation  function.  Second,  the  statistical 
properties  change  either  gradually  or  suddenly  to  various 
other  values  over  the  complete  range  of  conditions  for  which 
the  over-all  system  performance  is  to  be  optimized.  As  a 
result,  successive  measurements  of  those  properties  made  over 
that  range  of  conditions  are  found  to  differ  one  from  the 
other.  As  a simplified  example,  the  spectrum  of  a signal  in- 
put to  a servosystem  may  exhibit  a peak  at  one  frequency  dur- 
ing the  day  and  at  a quite  different  frequency  during  the  night. 
If  the  system  is  not  desired  to  have  a "day-night"  operation 
selector  in  order  to  be  optimum  during  each  period,  then  its 
overall  performs nos  must  be  optimized.  The  set  of  signal 
spectra  measured  for  day  and  for  night  is  a sufficient 
description  of  the  signal.  Those  spectra  constitute  the  values 
of  the  time-varying  signal  spectrum.  A time- varying  noise 
spectrum  must  have  the  same  characteristics. 

Time-varying  System  Parameters 

The  time-varying  system  parameters  may  include  aruplifioa- 
tions,  time  constemts  (electrical  or  mechanical),  eind  the 
like,  which  exhibit  random  changes  in  value.  These  random 
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changsc  may  be  tolerated  in  the  system  because  their  cause 
is  unknown  or  because  their  correction  is  not  feasible . 

They  may  result  fr'^m  internally  or  externally  caused  changes 
in  the  environment  of  the  system.  Some  examples  are  changes 
in  temperature,  pressure,  humidity,  and  power  sources. 

Another  possibility  is  fading  signal  reception.  These  random 
variations  may  also  bo  interpreted  to  result  from  inaccura- 
cies of  manufacture  of  some  system  part  where  the  problem 
is  to  optimize  the  total  of  the  performances  of  many 
similar  systems,  each  containing  such  a part. 

The  Scope  of  this  Dissertation 

There  are  two  major  parts  in  this  dissertation.  In 
Part  I,  the  spectrum  of  the  mean  square  error  -(the  error 
spectral  density)  is  derived  in  a manner  which  differs  in 
several  respects  from  that  found  in  the  literature  (Ic,  3)» 
The  form  of  the  spectrum  derived  herein  snows  promise  of 
being  more  useful  in  some  experimental  applications  than 
the  form  presented  in  the  literature,  for  a reason  explained 
later  in  this  paragraph.  The  signal  and  noise  inputs  to  the 
system  are  herein  described  by  the  amplitude  and  phase  com- 
ponents of  their  voltage  spectra  rather  than  by  their  power 
spectral  densities  as  is  commonly  doxie.  The  only  restric- 
tion is  that  the  signal  and  noise  enter  the  system  at  the 
same  point;  thiis  the  system  input  is  a signal  contaminated 
with  noise.  Correlv’tion  between  the  signal  and  noise  is 
herein  described  by  the  signal  and  noise  amplitude  spectra 
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and  a relative  phase  angle  which  refers  the  absolute  phase 
angle  of  the  noise  to  that  of  the  signal.  This  is  in 
contrast  to  the  use  of  cross-spectral  densities  in  the 
literature  (lo).  The  resulting  form  of  the  mean  square 
error  spectrum  derived  herein  may  be  often  more  useful  for 
experimental  application  than  that  presented  in  the  litera- 
ture for  the  following  reason.  The  cross  spectral  densities 
are  mathematical  concepts  involving  products  of  the  Fourier 
transforms  of  signal  and  noise  and  of  their  complex  conju- 
gates. Their  experimental  determination  probably  must  be 
accomplished  by  transformation  of  experimentally  measured 
cross-correlation  functions.  In  contrast,  the  relative 
phase  angle  may  generally  be  measured  directly,  as  described 
briefly  in  Part  I. 

In  Part  II,  the  mean  square  error  spectrum  is  used  to 
solve  the  optimization  problem,  especially  as  characterized 
by  random,  time- varying  behavior  in  the  input  spectra  and 

I 

system  parameters.  These  random  variations  are  treated  by 
operating  on  the  mean  square  error  or  on  its  spectrum  with 
a probability  theory  operation  which  has  been  termed  libcpeotea 
Value  in  one  reference  (4a)  and  Mathematical  Expectation  in 
another  (5a)»  This  operation  may  bo  applied  to  either  the 
form  of  the  mean  square  error  spectrum  derived  herein  or  the 
form  commonly  found  in  the  literature.  Use  of  this  operation 
to  allow  for  the  effects  of  the  random  variations  characteriz- 
ing the  problem,  rather  than  neglecting  such  random  behavior 
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permit’  the  design  of  syatemshaving  less  mean  square  error. 
Optimum  but  nonrealizable  system  funotlons  are  derived  for 
several  oases  of  random  time- variations.  A general  solution 
for  the  optimum  design  of  realizable  systems  of  a specified 
order  is  formulated  and  solved  for  six  important  cases. 

PART  I 

DERIVATION  OP  A MEAN  SQ.UARE  ERROR  SPECTRUM 

This  part  of  the  dissertation  contains  a derivation  of 
the  mean  square  error  spectrum.  For  the  purposes  of  this 
derivation,  the  signal  and  noise  Inputs  to  the  system  are 
represented  by  voltage  spectral  densities  rather  than  by 
power  spectral  densities,  a voltage  spectral  density  may 
be  considered  to  be  composed  of  an  amplitude  spectrum  and  a 
phase  spectrum,  these  Indicating  the  relative  potential  of 
each  frequency  component  present  and  the  phase  angle  of  each 
component.  Both  the  amplitude  and  phase  spectra  may  be 
determined  by  using  the  Fourier  transform  in  oases  where  the 
signal  may  be  described  functionally  in  time.  Otherwise, 
actual  measurement  using  a wave  analyser  or  like  Instrument 
will  yield  the  amplitude  spectrum.  The  phase  spectrum  may  be 
measured  in  an  analogous  manner  by  a modification  of  such 
an  instrument. 

The  input  to  the  system  to  be  optimized  is  assumed 
to  consist  of  some  signal  contaminated  with  noise.  A 
particular  frequency  component  of  the  signal  in  the  time-plane 


10 


may  be  written  as  the  real  part  of  a vector  rotating  in 
the  complex  plane  at  that  angular  frequency,  and  having 
for  its  magnitude  the  value  of  the  signal  amplitude  spec- 
trum jSm  . It  is  denoted  by 

(Ra- [ i(u)  . (1) 

The  CJ  component  of  the  noise  may  be  represented  similarly, 
with  the  addition  of  a relative  phase  angle,  ^(t iuf)  , which 
refers  the  phase  angle  of  the  co  component  of  the  noise  volt- 
age, in  the  time-plane,  to  that  of  the  signal  voltage.  The 
nature  of  the  relative  phase  angle  as  a function  of  time 
will  be  discussed  later  in  this  derivation.  The  cu  component 
of  the  noise  input  is  denoted  by 

hc<o)  (2) 

The  sum  of  the  signal  and  noise  vectors  represents  the  OJ 
component  of  the  system  input,  consisting  of  signal  oon- 
taminateA  with  noise.  It  is  written  as 

(L[  . ,3, 

The  system  output  spectral  density  is  the  result  of  the 
above  input  operating  on  the  system  transfer  function, 
denoted  by  the  familiar  M(u)  ^ , where  is  the 

closed  loop  gain  and  is  the  phase  shift.  A particular 

frequency  component  of  this  output  spectrum  is  written  as 

(PJ  Uicj)  £ lUu»  ] . (4) 

A frequency  component  of  the  error  is  the  difference  between 
the  »?ignal  and  the  output  components,  expressions  (1)  and 
(4)  above.  The  real  part  of  this  frequency  component  of 
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the  error  la 

6Ct  ; cj)  = Co5i<oi)  - M(4>)  Co5(o>t'+e<)  - 

ll(oJ)t^(fv)  Cod(o>t+«)  Cos(^Cticj)) 

Jl(eo)t^cco)  5{n(fit)t+e<)  Siri(^(t",io)),  (5) 

In  order  to  form  the  particular  frequency  component  of  the 
mean  square  error  spectrum,  the  error  of  that  component, 
equation  ($),  must  be  squared  and  averaged  over  Its  Indicated 
variations  as  a function  of  time.  The  magnitudes  of  the 
signal  and  noise  amplitude  spectra  and  the  system  function 
gain  and  phase  shift  are  held  temporarily  constant.  The 
Instantaneous  amplitude  of  each  component  of  the  signal 
and  noise  Inputs  was  defined  to  vary  In  time  as  described 
by  the  real  part  of  the  rotating,  complex,  vector.  However, 
no  definite  variation  In  time  was  assumed  for  the  relative 
phase  angle,  . If  the  signal  and  noise  are  complete- 

ly correlated,  this  angle  v;lll  have  a constant  value.  ?or 
zero  correlation  between  signal  and  noise,  the  variation  of 
the  angle  will  be  uniformly  random,  taking  on  all  values 
between  zero  emd  2Tfwith  equal  probabilities.  For  Inter- 
mediate degrees  of  correlation,  the  angle  will  vary  slowly 
about  some  average  value.  Because  the  nature  of  the 
variation  In  time  of  the  relative  phase  eingle  Is  markedly 
different  from  the  cyclic  variation  in  time  of  the  instanta- 
neous amplitudes,  these  two  functional  forms  may  be  assumed 
to  have  no  cross- correlation  in  their  variation  and  may  be 
avoraged  independently.  The  ave.raging  of  the  cyclic  varia- 
tions of  the  squared  error  spectrum  is  accomplished  by 
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integration  over  one  cycle.  The  time-average  of  the  cosine 
and  sine  of  the  relative  phase  angle  will  be  denoted  by  the 
conventional  bar  over  the  expression,  e.g. , Co^  . 

The  result  of  averaging  the  square  of  equation  (5)  is  thus 

6*  (o))  = ^ I )M*(h>)  - 

2 Mm  Cos + 

2 ^(co)  JU03)  Mm  [ MCcJ)- Cos  (c<m)]Co5( 

2.  M)  ?J M M M Siw6xc«>))  S in  (tj w)) },  ( 6 ) 

For  the  commonly  assumed  case  of  zero  correlation 
between  signal  and  noise,  the  variation  in  time  of  the 
relative  phase  angle  v/aa  noted  above  to  be  unlfcxrmly 
random.  The  averages  of  its  sine  and  cosine  values  are  in 
that  case  zero,  and  the  mean  square  error  spectrum  is 
accordingly  simplified.  For  other  values  of  correlation 
between  signal  and  noise,  the  averages  of  the  sine  and 
cosine  of  the  relative  phase  angle  are  not  intuitively 
known.  However,  they  may  be  determined  by  either  of  two 
ways,  depending  upon  the  form  of  the  information  available. 
If  the  probability  density  function  which  describes  the 
random  behavior  of  the  angle  is  known,  the  average  values 
may  be  oalculeted  by  use  of  the  Expected  Value  operation, 
explained  in  Part  II  of  this  dissertation.  If  the  values 
must  be  determined  experimentally,  the  angle  may  be  measured 
by  a modification  of  two  identical  wave  analysers  or  like 
instruments,  using  a common  oscillator  and  having  a rapidly 
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responding  phase  detector  coupled  between  them.  The  sine 
and  cosine  of  the  angle  may  be  determined  from  the  phase 
detector  output  by  a suitable  resolver,  and  their  averagrs 
may  be  read  on  two  appropriately  averagiiig  voltmeters. 

Because  ^ico)  and  7l(u>)  were  considered  to  be  the 
magnitudes  of  the  Input  signal  and  noise  amplitude  spectra, 
they  are  peak  values.  However,  amplitude  spectra  are 
commonly  given  or  measured  in  rms  values,  as  would  be 
obtained  by  taking  the  square  root  of  the  power  spectral 
densities.  Therefore  the  symbols  and  N(c^)  will 

henceforth  be  used  to  represent  tho  rms  values  of  the 
signal  and  noise  amplitude  spectra,  and  the  factor  of  1/2 
which  multiplies  equation  (6)  Is  removed.  The  mean  square 
error  spectrum  may  be  written  for  brevity  as 

e^(o>)=  S*+(S‘+W')M*'”2  5‘MCo5(«)-t- 
25NM  (M-  Coaw)  Cos(fl)  + 

2 5NM  5in(«)  Sinf^)  . 

By  proper  Interpretation  and  substitution  of  analogous 
symbols  (lc,3)t  this  spectrum  may  be  shown  to  be  equivalent 
to  that  commonly  found  in  the  literature.  The  prospect  of 
easier  application  of  the  mean  square  error  spectrum  derived 
above  to  many  experimental  studies  has  been  discussed  ear- 
lier. 
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PART  II 

THE  OPTIMIZATION  PROBLEM 
The  Method  of  Solution 

The  mean  square  error  spectrum  Is  a function  of  th?. 
signal  and  noise  amplitude  spectra,  the  averaged  sine  and 
cosine  of  the  relative  phase  angle,  and  the  system  gain 
and  phase  shift « If  any  exhibit  some  degree  of  random 
time-variation,  the  mean  square  error  spectrum  must  be 
averaged  over  all  of  their  possible  values  to  form  the 
expected  or  most  probable  value  of  that  spectrum.  It  Is 
this  expected  mean  square  error  spectrum  that  represents 
the  over-all  performance  of  the  system  and  Is  to  be 
minimized  by  proper  choice  of  the  system  gain  and  phase 
shift.  This  process  of  averaging  the  spectrum  over  all 
r€Uidom  values  of  the  Input  spectra  and  system  function  Is 
accomplished  by  the  use  of  a probability  theory  operation 
termed  Expected  Value  (4e)  or  Mathematical  Expectation 
( $a) . Appendix  A of  this  dissertation  contains  a descrip- 
tion of  an  experimental  check  on  this  use  of  the  Expected 
Value  operation. 

A limitation  on  the  rate  of  random  time -variation 
results  from  the  separation  of  the  averaging  process  Into 
two  steps.  The  first  is  the  time  average  of  each  frequency 
component  of  the  squared  error  spectrum  over  one  cycle  of 
its  angular  frequency.  The  second  step  is  the  Expected 
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Value  operation,  averaging  each  frequency  component  of  the 
moan  square  error  spectrum  over  all  random  variations  in 
the  input  spectra  and  system  parameters.  Therefore  the 
reuidom  variation  must  be  slow  enough,  compared  to  the  angular 
frequency  of  the  component  being  averaged,  that  the  separation 
of  the  averaging  process  is  valid. 

Since  the  kind  of  random  time-variations  referred  to  here 
generally  occur  very  slowly,  or  at  infrequent  intervals,  this 
limitation  may  be  assumed  to  be  met. 

The  Expected  Value  of  a function  of  any  number  of  random 
variables  is  defined  (4a)  as  the  average  value  of  that  func- 
tion when  its  random  variables  take  on  all  of  their  permis- 
sible values  with  their  associated  probabilities  of  occur- 
rence. Briefly,  the  Expected  Value  of  a function,  f(x)  , 
of  the  random  variable,  X , where  X takes  on  each  value 
idth  a probability  p(x)  , is  denoted  by  El[f(x)]  . Either 
it  is  determined  mathematically  by  a}  summation: 

EI[ffx)]  = ^ f(K)  p(x)  » ^ takes  on  only  discrete 

values  in  which  case  p (X)  is  termed  the  discrete  probablJity 
density  function  of  X ; or  it  is  determined  by  b)  integra- 
tion: E[fw]=  „-fW  p(>c)-dx  , if  ^ varies  continuously 
in  which  case  p(x)  is  termed  the  probability  density  func- 
tion of  X . Eor  all  functions  of  any  practical  importance 
(5a),  the  following  properties  of  the  Expected  Value  operation 
are  valid.  The  Expected  Value  of  a sum  is  the  sum  of  the 
Expected  Values.  The  Expected  Value  of  a constant  is  that 
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constant.  The  Expected  Value  of  a product  of  functions  of 
statistically  independent  random  variables  is  the  product  of 
their  separate  Expected  Values.  The  Expected  Value  of  a func- 
tion, f (x,y,z)  , of  several  random  variables,  X , y , and  Z , 
having  a joint  probability  density  function,  p(X,y,Z)  , is 

E[f(X,y,z)]=  ///,!,  2 p(x,ij,z)  dx  dy  dr  . 

The  requirements  for  a probabil.'' ty  density  function,  p(x)  , 

are  that  p(x)- 0 and  that  J pCxjdx  =1  since  it  is  certain 
that  X will  take  on  some  value  within  its  range  of  values. 

As  mentioned  above,  any  or  all  of  the  quantities  compos- 
ing the  mean  square  error  spectrum  may  exhibit  random 
time-variation.  In  addition,  the  variaticuis  of  any  quantity 
may  bo  dependent  in  the  probability  sense  on  the  variations 
of  any  or  all  of  the  other  quantities.  Because  of  the  many 
possibilities  for  such  varied  and  interrelated  random  beha- 
vior, the  method  of  solution  will  be  presented  in  the  form 
of  several  cases.  For  each  case,  a different  set  of  restric- 
tions on  the  many  possibilities  for  dependent  and  independent 
random  behavior  will  be  used.  These  cases  are  divided  into 
two  groups.  The  division  results  from  two  possible  procedures 
in  using  the  mean  square  error  spectrum,  as  la  explained  in 
the  next  paragraph. 

The  first  prr oedure  involves  minimization  of  each  fre- 
quency component  (or  ordinate)  of  the  mean  square  error  spec- 
trum. There  results  the  optimum  system  transfer  function  in 
terms  of  the  input  spectra.  The  second  procedure  requires 


- 17  - 


minimization  of  the  total  mean  square  error,  which  has  been 
shown  In  the  literature  to  be  the  Integral  of  the  mean  square 
error  spectrum  over  all  real,  positive  frequenoies.  Each 
procedure  has  Its  special  advantages  and  disadvantages.  The 
first  procedure  yields  generalized  results  in  a relatively 
straightforward  manner.  The  second  procedure  permits  a 
generalized  formulation  of  Its  solution.  However,  to  achieve 
useful  results  the  Integration  over  all  frequency  requires 
the  substitution  of  the  typo  of  system  transfer  function  to 
be  optimized.  Specific  signal  and  noise  amplitude  spectra 
and  the  averages  of  the  sine  and  cosine  of  the  relative  phase 
angle  must  also  be  substituted.  These  useful  results  are  the 
optimum  values  of  the  parameters  of  the  type  of  system  substi- 
tuted, given  in  terms  of  the  coefficients  of  the  spectra 
chosen.  Thus  the  second  procedure  assures  physically  realiz- 
able, optimum  systome.  However,  the  first  procedure  will  be 
shown  to  specify  that  the  optimum  system  has  a gain  charac- 
teristic determined  by  the  shapes  and  relative  magnitudes  of 
the  input  spectra  and  a phase  shift  characteristic  every- 
where zero.  Such  a system  is  seldom  physically  realizable 
because  it  is  generally  Impossible  to  have  a rate  of  change 
of  attenuation  v/ith  respect  to  frequency  without  having  non- 
zero phase  shift.  In  fact,  for  minimum  phase  shift,  stable, 
linear  systems  composed  of  fixed,  lumped  elements,  Bode’s 
theorms  ( 2a  ) and  others  indicate  that  the  phase  shift  at 
any  frequency  is  a unique  function  cf  the  attenuatjon  at  all 
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froquenoies.  In  spite  of  this  limitation  the  generality  of 
the  first  procedure  yields  valuable  qualitative  insight  into 
the  synthesis  problem.  In  addition,  by  yielding  the  optimum 
though  nonrealizable  system  transfer  function,  it  may  be  used 
to  calculate  the  lower  bound  of  the  mean  square  error  for  any 
particular  input  spectra.  This  will  be  a limit  which  the 
mean  square  error  of  physically  realizable  systems  may 
approach. 

Recognizing  the  importance  of  both  of  the  procedures 
described  above,  this  dissertation  will  treat  the  procedures 
separately  in  the  two  major  sections  that  follow.  These  are 
"Optimum,  Idealized  System  Transfer  Functions"  and  "Optimiza- 
tion of  Physically  Realizable  Systems".  As  mentioned  above, 
each  section  will  include  several  oases  which  illustrate  the 
solution  for  problems  characterized  by  various  types  of 
random  time-variation. 

Optimum.  Idealized  System  Transfer  Funotions 

Independently  Time- varying  Input  Spectra.  For  the  solu- 
tion of  a case  which  illustrates  both  the  first  procedure  for 
using  the  moan  square  error  opeotrum  and  the  treatment  for 
random  time-variations,  the  following  assumptions  and  restrlo- 
tlons  are  made.  The  signal  and  noise  amplitude  spectra  are 
assumed  to  be  Independently  time-varying,  having  probability 
density  funotions  p(5(w)}  and  cj(W^a»>)  . Each  density  func- 
tion describes  the  probability  of  occurrence  of  each  amplitude 
that  a particular  frequency  component  (to. )of  the  time- varying 
spectrum  exhibits  for  the  set  of  measurements  of  the  spectrum. 
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Those  probability  deasity  funotions  may  be  any  one  of  the 
familiar;  Uniform,  Gaussian  or  Normal,  Gamma,  Beta,  or  they 
may  be  some  undefined  density  funotlon.  As  will  be  shown, 
the  shape  of  the  density  funotlon  need  not  be  known;  only 
its  first  and  second  moments  are  used  In  mean  square 
minimization.  The  assumption  of  Independence  between 
signal  and  noise  Implies  the  restriction  that  their  relative 
phase  angle  has  a uniform  probability  density,  taking  on  all 
values  from  zero  to  271  with  equal  probabilities.  Thus  the 
averages  of  the  sine  and  cosine  of  in  the  mean  square 

error  spectrum  of  equation  (7)^  are  zero.  The  system  funo- 
tlon Is  restricted  to  be  time- Invariant  for  the  case  follow- 
ing. The  Expected  Value  of  the  mean  square  error  spectrum, 
equation  (7)i  under  these  assumptions.  Is 

The  various  Expeoted  Values  may  be  evaluated  as  follows: 

L>S  piS)dS=^,  , E[5*]  = (.„3S'-p(;s)cI5=^,  , 

N q(N)  dN=n,  , EIIN']=j.„ ^ = , 

where  , and  71,  are  the  averages  pr  first  moments,  and 
and  *>12  ure  the  second  moments.  The  second  moment  Is  often 
written  in  terms  of  both  the  first  moment  and  the  variance. 
The  variance,  denoted  by  0“^  , Is  the  second  moment  about 
the  mea'' , and  Is  a measure  of  the  spread  of  a probability 
density  function  about  its  mean  value.  The  positive  square 
root  of  variance,  cr* , is  commonly  termed  the  standard  devia- 
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tlon.  ?or  a random  variable  X with  a mean  value  X,  , the 
variance  cf  x ie 

«*=>£:  [(x-x,)’]  = E[x*]“  2 X,  E[x]+X,‘  = Xa-  X,*. 
Therefore!  E [ S']  = ^* ■(•  UJ*  , and  E[N*]=  7!,*+i>5'*' 

These  moments  are  functions  of  frequency,  being  spectra 
describing  the  sets  of  signal  and  noise  amplitude  spectra. 
Substituting  these  evaluations  in  the  expected  mean  square 
error  spectrum  yields  £ [”e*(n>)]=  (^♦*+05'*)  + 

M*(  + 2 + MCoeCof).  (9) 

These  moments  are  of  special  interest  because  they 
indicate  the  information  which  is  ^lecessary  for  the  use 
of  the  moan  square  criterion.  The  first  moment  is  the 
average  of  the  amplitude  spectra  which  describe  the 
time- varying  spectra.  The  second  moment  is  the  average  of 
the  squares  of  the  amplitude  spectra  (the  mean  square);  it 
is  thus  the  average  of  the  power  spectra  describing  the 
signal  or  noise  time-varying  spectrum  (the  first  moment  of 
the  power  spectra).  The  first  and  second  moments  of  the 
amplitude  spectra  constitute  all  the  information  about  the 
time -variation  in  input  spectra  or  system  function  that  the 
mean  square  criterion  requires.  No  other  information  can 
be  used.  For  this  case  of  uncorrelated  signal  and  noise, 
the  method  requires  only  the  average  of  the  power  spectra; 
the  first  moments  of  the  amplitude  spectra  are  not  required. 
The  averages  referred  to  above  are  ail  weighted  according  to 
the  probabilities  of  occurrence  of  the  various  spectra,  as 
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the  Expected  Value  operation  indioatea.  vVriting  the  second 
moment  ot  the  amplitude  spectra  as  the  square  of  their  first 
moment  plus  their  variance  permits  using  the  variance  term 
to  Illustrate  the  effect  of  random  tlme>variatlons  in  the 
input  spectra. 

The  synthesis  problem  in  this  case  is  the  determination 
of  the  system  gain  and  phase  shift  as  functions  of  the  signal 
and  noise  input  spectra  to  xoinimize  the  expected  mean  square 
error  spectrum.  The  phase  shift  (ot)  at  a particular  fre- 
quency is  a unique  function  of  the  gain  (M)  at  all  frequen- 
cies for  nearly  all  networks  and  systems  (ie.  for  minimum 
phase  shift,  linear,  stable  systems  composed  of  fixed, 
lumped  elements),  as  expressed  by  Bode  (2a  ) and  others. 

If  this  function  could  be  substituted  in  the  error  spectrum, 
the  derivative  with  respect  to  M could  be  equated  to  zero  to 
determine  the  optimum  M.  From  this  the  optimum  o(  wouldbe 
calculated.  Since  the  relationship  between  M and  ok  requires 
treatment  of  all  frequencies  concurrently,  and  is  so  in- 
tractable as  to  prevent  even  that  use  of  it,  the  classical 
assumption  of  independent  of  M will  be  made.  Therefore  a 
much  less  restricted  system  function  will  be  optimized. 

It  includes  all  physically  realizable  systems  as  a special 
case.  The  resulting  gain  and  phase  shift  characteristics 
will  be  the  best  possible  although  probably  not  physically 
realizable.  The  assumption  of  Independence  between  o<  and  M 
allows  the  partial  derivatives  of  any  component  of  the 
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spectrum,  tairen  with  respect  to  ok  and  M , to  be  each  equat- 


ed to  zero  and  solved  for  their  optimum  values. 

h E ['^(<o)]  /da.  = Z Sir\  (oc)  = 0 , 

which  requires  that  oisO  ; that  is,  a zero  phase  shift 
system  is  indicated  to  be  optimum. 


iE [■?(«)] /iM  = 2M  U.  + >i»)-2  42  Cos4x)=  0 

yields,  with  o(,aO,  Ul) 


If  the  second  moments  are  denoted  by  F|  and  , the 
averages  of  the  seta  of  signal  and  noiae  power  spectra, 

P 

then 

which  result  has  been  developed  by  Wiener  (6)  in  a basical- 
ly different  manner. 

Writing  the  second  moment  as  the  square  of  the  first 
moment  plus  the  variance  permits  using  the  variance  term 
to  illustrate  the  effect  of  random  time-variations  in  the 
input  spectra.  From  equation  (11),  the  gain  of  the  optimum, 
zero  phase  shift  system  transfer  function  is 

The  latter  form  indicates  ohat  the  effect  of  time-vary- 
ing behavior  in  a particular  frequency  component  of  the 
signal  spectrum  is  to  increase  the  optimum  value  of  closed 
loop  system  gain  toward  unity  at  that  frequency.  The  effect 
of  time-varying  behavior  in  the  noise  spectrum  is  conversely 
to  decrease  the  closed  loop  gain  toward  zero.  It  is  inter- 
esting to  note  that  the  zero  frequency  closed  loop  gain  of 
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the  system  should  not  necessarily  be  unity  as  is  generally 
seen  in  the  literature.  Consideration  of  the  expression 

M (O)  = I /(  I + '71^(0) /Axlo)) 

yields  the  oonoluslon  that  the  optimum  zero  frequency  gain 
depends  upon  the  limit  of  as  CO  approaches 

zero.  That  gain  will  lie  between  unity  and  zerO|  the  end 
points  being  special  oases  of  the  noise  or  signal  spectra 
alone  being  zero  at  zero  frequency.  Of  course  the  analogous 
oonoluslon  la  valid  at  any  frequency.  Note  that  an  optimum 
value  of  closed  loop  galn^M,  greater  than  unity  is  never 
Indicated. 

The  oharaoteristio  of  an  optimized,  physically  realiz- 
able system  transfer  function  will  tend  toward  the  non- 
reallzable,  idealized  oharaoteristio  specified  in  this 
section.  However,  the  realizable  design  will  necessarily 
involve  compromises.  It  must  smooth  out  any  sharp  peaks 
or  valleys  in  the  idealized  gain  oharaoteristio  as  a result 
of  the  inherent  phase  shift  and  fixed  shape  of  any  type  of 
realizable  system  transfer  function. 

The  next  two  oases  will  constitute  a sufficient 
illustration  of  the  method  as  applied  In  this  fjrst  section 
of  Part  II  of  the  dissertation.  In  the  next  case  the  signal 
and  noise  spectra  will  be  assumed  to  be  dependently  time-vary- 
ing .In  the  final  case  the  system  function  will  be  assumed  to 
exhibit  some  independent,  random,  time-varying  behavior,  and 
the  input  spectra  will  be  assumed  to  be  time - invariant . 
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Dependent ly  Time -varying  Input  Spectra.  In  this  the 
second  case  of  Part  II,  the  signal  and  noise  oiaplitude 
speotra  are  asaumed  to  bo  depondently  time-varying,  having 
therefore  a Joint  probability  density  function  *f(6)N)  . 

The  average  values  of  the  sine  and  cosine  of  the  relative 
phase  angle  are  also  assumed  to  vary  in  time,  independently 
of  the  amplitudes  for  simplicity.  The  probability  density 
for  the  averaged  sine  is  denoted  by  p(Sin  ? ).  It  is  defined 
for-i^  and  is  assumed  to  be  an  even  function  about 

zero  with  its  most  probable  value  at  zero  also.  The 
probability  density  for  the  averaged  cosine  is  similarly 
denoted  by  cjCcos^)*  It  is  assumed  to  have  its  most  probable 
value  at  plus  one.  The  system  function  is  to  remain 
time -invariant  for  this  case.  The  form  of  the  expected  mean 
square  error  spectrum  under  these  assumptions  is 

E[5*]  + (E[S'J+EW'j)M‘- 

2 EfS']  M Cos(«)  + 

2 E[5N]M 

2E[SNjM  5in(«)  ElSin^j.  (12) 

The  Expected  Values  of  5*  and  N*  are  determined  in  the  same 
manner  as  was  given  in  the  preceding  case. 

E[S*j=-42=-^,“+<jr*  , E.[N“J= 

where  and  ti,  are  the  first  moments,  ^ and  7li  are  the 
second  moments,  and  05^  and  c^^are  the  variances. 

An  evaluation  which  did  not  appear  in  the  previous  case  is 

EIsn]  = 5N  fCS,N)  c(3  dW=^  f <r,Oi 

where  P is  termed  the  correlation  between  S and  , . 
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If  p is  zero,  S and  N are  said  to  be  independent  and  the 
above  Expected  Value  would  be  obtained  simply  by 

E[SN]  = E[S]*E[Nj~^.'n,  . 

The  Expected  Values  of  the  averaged  sine  and  cosine  are 

E [ 5in  ^ 3 = Sin  ^ • p(5in  * diSin^)  = O 

because  of  the  assumed  nature  of  p(5f«  , and 

El  Cos C05^  • cj(Coi7) ' ^ f 

which  is  assumed  to  have  some  value  between  ze3*o  and  unity, 
depending  upon  the  shape  of  Cj ( dss  . Let  that  value  be 

denoted  by  P . Substitution  in  the  expected  mean  square 
error  spectrum,  equation  (12),  yields 

E[?M]=  (^*+05^)+  d3» 

+ M Co5(o()  4- 

2 (^,72,  +fd505)  M(M“  Co3(o<))  P, 

Partial  differentiation  as  before  yields 

2 (^.>»,4*?05  0'„)’M  Sm(o<)  p = 0. 

This  requires  that  either 

U.’  + Os')  + P(-4.>t, = 0 

or  SIm(oO=o  and  therefore  o<>«o  at  all  frequencies.  Set- 
ting the  partial  derivative  with  respect  to M equal  to  zero 

M(w)  = ‘?1E CosM 

(^,*+asO  + (V+q,*)  + 2.U,yi,4-fCi<r,,)P  , (13) 

The  first  condition  on  o{  mahes  the  closed  loop  gain  equal 

zero.  Therefore  the  mean  square  error  would  be  as  large  as 

if  there  were  no  syste^i.  Tho  other  condition,  , 
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assures  a minimum  of  the  error  with  respect  to  oc  for 

-f  (Tj^  ) + ( >i<  + f ) P > O . 

Although  f may  range  between  minus  and  plus  one,  the  in- 
equality will  generally  be  satisfied.  For  this  condition, 
the  optimum  system  has  zero  phase  shift  everywhere  and  the 
gain  oharaot eristic,  M (cj)  , given  by  equation  (13)  with 
oC  at  0. 

Time-varying  System  Closed  Loop  Gain.  In  the  final 
case  of  this  section,  the  system  transfer  function  is 
assumed  to  exhibit  some  Independent,  random,  time- varying 
behavior.  The  phase  shift  is  assumed  to  be  everywhere  zero. 
The  gain  is  assumed  to  have  a probability  density  function 
p{M)  . The  signal  and  noise  are  assumed  to  be  uncor- 
related  and  their  amplitude  spectra  are  time -invariant. 

The  expected  mean  square  error  spectrum  is  therefore 

E[Fm]  = S*  + (s‘+  N*)E [M‘] - 2 5'  E[M]  . 

The  required  Expected  Values  are 

= fall  M M'p(M)’d’M=  >n,  , 

“ j.||„  T7t,‘+  crs;*. 

The  optimization  problem  is  to  determine  the  best  value  of 
yn, , the  average  of  the  randomly  varying  system  closed  loop 
gain,  M(Ct’)  . The  variance  of  that  random  variation  is 
aasumevl  to  be  a fixed  fraction  of  the  average.  Therefore 

>n,*  + -t  W S*  + . 

This  is  more  reasonable  than  to  assume  a variance  which  is 
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Indopsndent  of  tho  average,  whether  that  average  is  large 
or  small.  Substitution  In  the  expected  mean  square  error 
spectrum  7lelds 

E[Pfo))]  = 5*+(f’+N*)7n,’0H*) -as'w.,  , (u) 

Minimization  of  this  spectrum  with  respect  to»iiby  setting 


its  derivative  equal  to  zero  yields 


yn,= 


:s'+N*)(i+i') . (15) 


A conclusion  evident  here  is  that  random  variation  in  the 
closed  loop  system  gain  dictates  a smaller  value  of  average 
gain  that  would  otherwise  be  designed.  Another  conclusion 
is  of  more  importance;  it  may  be  shown  that  if  the  random- 
ness of  the  gain  were  neglected,  a larger  value  of  mean 
square  error  would  alw^s  result.  To  demonstrate  this 
conclusion,  a ratio  of  two  evaluations  of  the  expected  mean 
square  error,  equation  (14)  above,  is  formed.  The  ratio 
is  the  evaluation  using  the  value  of  Tn, derived  above 

(equation  (15))  by  properly  treating  the  randomness  of  the 

S* 

gain,  divided  by  the  evaluation  using  771,  = 'grp’yjT 
which  results  from  treating  the  gain  as  a nonrandom  parameter. 

I 

The  rat5o  is  rTca — ^ 

1 4.  9 O 

Thus  the  coaoiusion  immediately  above  is  demonstrated  by 
this  ratio  being  always  less  than  unity,  except  in  the  limit 
of  S or  equal  to  zero.  If  S-0,  there  is  no  signal  input 
and  both  values  of  gain  are  zero,  hence  both  have  the  same 
error.  If  6=0,  there  is  no  randomness  and  both  values  of 
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gain  are  the  same,  henoe  the  errors  are  the  same. 

Optimization  of  Physically  Realizable  Systems 

Thus  far,  application  of  this  method  has  been  restrict- 
ed to  systems  having  zero  phase  shift.  Recognizing  the 
general  impossibility  of  a rate  of  change  of  attenuation 
vidthout  phase  shift  in  physically  realizable  systems,  the 
second  procedure  described  previously  will  be  considered  in 
detail  in  this  section.  The  most  advantageous  i\se  of  this 
more  realistic  procedure  lies  in  substitution  of  a particu- 
lar type  of  system  transfer  function  for  the  general 
expression,  in  the  mean  square  error  spectrum. 

Particular  forms  of  the  signal  and  noise  amplitude  spectra 
are  also  substituted.  The  procedure  involves  Integrating 
the  mean  square  error  spectrum  over  all  real,  positive 
frequencies  to  form  the  mean  square  error.  For  problems 
characterized  by  random  time -variations  in  the  input  spectra 
and  system  transfer  function,  the  average  of  the  mean  square 
error  over  those  variations  must  bo  determined.  This  is 
accomplished  by  the  use  of  the  Expected  Value  operation  as 
described  and  applied  earlier.  The  problem  of  system  op- 
timization is  that  of  minimizing  the  expected  mean  square 
error  with  respect  to  such  system  parameters  as  amplifica- 
tion and  time  constants,  within  the  limitations  of  system 
stability.  Such  minimization  is  carried  out  by  ordinary  or 
partial  differentiation  or  by  graphical  means.  There  results 
the  ootimura  values  of  the  system  parameters  and  the  optimum 
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characteristic  of  the  type  of  system  transfer  function 
substituted.  The  conclusions  for  the  case  of  independent, 
random  time-variations  In  the  Input  spectra  are  much  the 
same  as  given  for  the  first  procedure.  These  are:  random- 
ness In  the  signal  spectrum  dictates  an  Increase  In  system 
gain  for  optimization,  while  randomness  In  the  noise  spec- 
trum dictates  a decrease  In  system  gain.  The  second  moments 
of  the  amplitude  spectra  as  described  before  will  dictate 
the  proper  values  of  system  parameters.  The  conclusions  for 
oases  of  randomness  In  the  system  transfer  function  are 
determined  by  this  second  procedure  In  a different  manner. 

In  contrast  to  simply  attributing  the  randomness  to  the 
entire  closed  loop  gain  function  as  was  necessary  under  the 
first  procedure,  this  second  procedure  permits  ascribing 
the  randomness  directly  to  one  or  more  specific  system  para- 
meters. Examples  of  such  random  variation  are  drifting 
amplifications  and  fluctuating  time  constants.  As  a result 
of  this  contrast,  the  cases  of  random  time- variation  In  this 
section  will  be  restricted  to  randomness  in  system  time 

ooiist'anta. 

The  general  solution  to  the  optimization  problem  as 
described  above  Is  formulated  below  In  iaathamatioal  terms. 
Illustrating  this  second  procedure  symbolically.  Pai^tioular 
solutions  for  some  restricted  cases  are  obtained  later  in 
this  section.  For  the  general  formulation,  the  input  signal 
and  noise  amplitude  spectra  are  denoted  by  products  of 
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power  eerie s:in  cO  ; 

These  Input  spectra  operate  on  a system  transfer  function 
given  in  general  by: 

€'^  *"  JT  ( + j ^ ) y TT(  , for  vrh  ich 

M(W)  = IT  -/  f k* + + ""  V ‘ and 

«M=  g tan'TJ^)  - E tan"(:^). 

The  general  mean  square  error  spectrum,  6*(Ji>)  , is  the 
result  of  substitution  of  the  terms  above  in  equation  (7). 

The  formulation  of  the  general  solution  is  a set  of  simul- 
taneous equations  > _ - ^os 

dci,E-l-  i -O, 

Where  is  successively  each  of  the  system  parameters 
ifk,K  ,’X,,Ty  ) which  are  considered  to  be  subject  to 
adjustment.  The  resulting  values  of  system  parameters 
must  satisfy  the  restriction  that  N*l 

TT  }(<>T,)-o 

V»l 

have  no  roots  with  positive  real  parts,  thus  insuring  system 
stability.  The  zero  of  each  partial  derivative  must  be 
examined  to  instire  that  it  represents  a minimum  of  mean  square 
error  ather  than  a maximum.  The  Expected  Value  operation  is 
carried  out  over  all  system  function  and  input  spectra  para- 
meters. The  random  behavior  of  these  pareuneters  is  described 
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in  general  by  a joint  probability  density  function 
p(/3k  -I  Sv  ) yik  i 5 

An  alternate  and  i&ore  compact  formulation  may  be  developed 
from  the  concept  of  a system  space  as 

grad  E [ £ e*(tt»>eiai  ]=  o. 

The  formulation  above  has  been  continued  to  completion 
for  soma  specific  classes  of  problems.  These  solutions  are 
described  below.  The  general  solution  has  not  been  treated 
further  in  this  dissertation  because  further  general  treat- 
ment is  not  feasible. 

?or  the  first  class  of  problems  below,  this  procedure 
is  applied  to  a set  of  time -Invariant  servosystema.  The 
approach  in  that  section  is  somewhat  more  general  than  that 
found  in  the  literature  (lb).  A discussion  of  the  solutions 
Illustrates  some  of  the  conclusions  resulting  from  the  use 
of  the  mean  square  criterion.  The  Invariant  class  is  pri- 
marily Included  as  an  introduction  to  the  later  classes, 
which  are  characterized  by  random,  time-varying  system 
tioM  constants. 

Optimization  of  Invariant  Systems.  There  is  described 
in  this  part  of  the  paper  the  application  of  the  procedure 
just  formulated  to  three  general  types  of  servosystems , 

The  signal  amplitude  spectrum  is  assumed  to  be  of  first 
order  form,  as  shown  in  figure  1.  A white  noise  spectrum 
is  assumed.  The  correlation  between  signal  and  noise  is 
assumed  to  be  zero.  The  signal  to  noise  ratio  is  described 


I 


PiQure  I 
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in  terms  of  the  values  of  the  signal  and  noise  amplitude 
spectra  at  zero  frequency  (do).  The  ratio  is  denoted  by  2T 
and  is  an  important  parameter  in  the  examples  to  follow. 

The  series  of  general  servosystems  to  be  optimized  may  be 
described  by  their  open  loop  transfer  functions,  which 
are  (figure  2);  (1)  an  integrator,  (2)  a simple  lag,  and 
(3)  a simple  lag  with  an  ideal  amplifier. 

For  each  example  of  the  series,  the  input  spectra 
shown  In  figui’e  1 and  the  system  gain  and  phase  shift  are 
substituted  in  the  proper  forii  of  the  mean  square  error  spec- 
trum. 

The  spectrum  is  Integrated  over  all  real  frequencies, 
by  the  method  of  residues,  to  form  the  meein  square  error. 

The  values  of  the  system  parameters  which  minimize  the 
mean  square  error  for  a given  ratio  of  signal  to  noise  are 
determined  by  ordinary  or  partial  differentiation.  These 
values  are  subject  to  the  requirements  of  system  stability. 
The  parameters  of  these  three  systems  are,  for  system  (1) 
the  Integration  constant  T,  (2)  the  time  constant  T , and 
(3)  the  gain  K and  time  constant  T.  Ordinary  differentia- 
tion is  used  to  optimize  the  first  two,  single -parameter  sys- 
tems. In  more  complex  examples  of  two-parameter  systems, 
the  partial  differentiation  becomes  too  intractable,  and 
the  surface  generated  by  the  mean  square  error  as  a func- 
tion of  the  two  parameters  is  bettex*  examined  graphically 
for  its  minimum.  The  mathematics  involved  In  this  applies- 
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tion  is  straightforward  in  essence . 

As  a more  detailed  illustration  of  the  method,  the 
essential  points  in  the  optimization  of  the  first  system 
are  presented  here.  Ths  signal  and  noise  amplitude  spectra 
are  given  by 


The  closed  loop  system  transfer  function  of  the  integrator  is 

" l + JwT  ' '■  ~ ^i  + co*r*‘  ) 

=c=-tan-Yo.T)  = 

These  quantities  are  substituted  in  the  mean  square  error 
spectrum,  equation  (7),  which  is  written  here  for  zero  cor- 
relation between  signal  and  noise. 


5*+  (S*+N*)M'‘-2S*MCos(«). 

Substitution  yields 

'^2  - — /3* ^ . I , yiy 

« w - "|  + iJ*  r*  l+h)*T*  . 

Integration  by  residues  results  in  the  mean  square  error, 

® Z IJ  {(HSr)  T j, 

which  may  be  written  in  terms  of  the  signal  to  noise  ratio, 

Tl“  AS  ^ n:  / 1 ~ I u.  ^ I (17) 

u.  c a ^ l‘  lisr  ^ z»srj. 


hsr  ' z»5' 

Minimization  of  this  escpression  is  accomplished  by  setting 
the  deivatlve  of  8*  with  respect  to  'T  equal  to  zero  and 
solving  for  the  optimum  T , within  the  system  stability 
requirement  that  C be  nonnegatlve.  This  yields 


+ 1 


The  first  root  yields  a maxl- 


mum  of  and  an  unstable  system.  The  second  root  yields 
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a minimum  of  ^ aM  honoe  is  the  optimum 

time  constant.  This  result  v/ill  be  discussed  below. 

The  results  of  optimizing  the  three  types  of  servo- 
systems  listed  above  are  presented  in  figure  3>  The  mean 
square  error  of  each  optimized  system  is  shown  as  a func- 
tion of  the  signal  to  noise  ratio  !£  . The  mean  square  error 
function  for  the  zero  phase  shift  system  previously  derived, 
equation  (11),  is  included  for  comparison,  despite  the  un- 
realizability of  that  system,  because  it  represents  a lower 
bound  on  the  mean  square  error  attainable.  These  functions 
are  normalized  to  the  mean  square  error  of  a no-pass  system 
(a  system  having  no  output).  Fl)e;ure  3 Illustrates  several 
important  conclusions  of  interest  for  both  system  analysis 
and  synthesis.  These  are: 

(1)  A worthwhile  Integrator  system  cannot  be  designed 
for  23  < 1 : all  stable  Integrator  systems  have  at  least  as 

much  error  as  the  no-pass  system  for  any  such  sigma.  The 
reason  is  that  the  integration  constant  yielding  minimum 


mean  square  error,  given  by 


I 


becomes  infinite 


as  23  approaches  unity,  that  is,  the  system  bandwidth  becomes 
zero.  For  12  < I , the  optimum  T becomes  negative,  hence 
the  system  is  unstable,  and  any  positive  T yielda  more  error 
than  a no -pass  system. 

(2)  The  mean  square  error  curve  of  the  simple  lag 
system  is  similar  to  that  of  the  integrator.  The  optimum 


time  constant  is  given  by  T = — [j 


which  becomes  in- 
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finite  when  5T  = Vff  • 5’or  emailer  ^ the  system  is  worth- 
less. The  system  has  a gain  of  1/2  at  zero  frequency.  That 
this  is  desirable  for  T,  near  unity  is  shown  by  its  super- 
iority to  the  integrator  system  which  has  a gain  of  unity 
at  zero  frequency.  Por  H larger  than  3*7  the  integrator 
system  is  seen  to  be  superior,  indicating  the  desirability 
of  unity  gain  at  zero  frequency  for  large  signal  to  noise 
ratios . 

(3)  The  simple  lag  with  an  ideal  amplifier  is  seen  to 
be  superior  to  the  first  two  systems  almost  everywhere. 

This  superiority  may  be  attributed  to  the  gain  parameter 
which  permits  changing  the  zero  frequency  gain,  and  hence 
the  entire  level  of  system  gain,  according  to  the  ratio  of 
signal  to  noise.  At  the  point  where  71 , this  system 
coincides  with  that  of  system  (2)  because  its  optimum  gain 
is  unity.  Below  its  optimum  gain  is  less  than 

unity;  the  amplifier  is  only  an  attenuator.  In  this  region 
of  23  , system  (2)  could  bo  improved  easily  by  the  ad- 

dition of  an  attenuator  in  its  open  loop,  making  it  the  same 
as  system  (3) . 

(4)  In  addition  to  the  systems  described,  a second 
order  system  was  similarly  treated.  It  has  a closed  loop 
transfer  function  composed  of  the  product  of  two  simple  lag 
oirouits  with  independent  time  constants.  Optimization  of 
this  system  with  respect  to  the  two  time  constants  dictated 
that  one  of  them  be  zero  for  minimum  mean  square  error. 

Thus,  under  the  assumptions  rmade,  the  best  second  order 
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aystom  of  this  form  is  a simple  first  order  system  corres- 
ponding to  the  integrator  system  treated  first. 

The  optimization  procedure  for  the  two-parameter 
systems  discussed  above  as  (3)  and  (4)  is  described  In 
detail  in  Appendices  B and  C of  this  dissertation. 

Optimization  of  Random  System  Parameters.  In  the 
preceding  class  of  problems,  the  servosystem  was  considered 
to  be  composed  of  fixed  elements,  that  is  of  elements  such 
as  gain  eind  time  constants  having  constant  values  during 
the  period  of  operation  for  which  the  system  was  being  de- 
signed. It  is  possible  that  the  environment  of  the  system 
may  vary  greatly  during  that  period.  As  a result  of 
changes  in  pressure,  temperature,  humidity,  and  the  like, 
the  gain  and  time  constants  may  vary  appreciably  from  their 
intended  values.  Such  variations  are  of  more  or  less  ran- 
dom nature. 

For  this  case  the  system  gain  ( N ) and  phase  shift  ( o< ) 
are  assumed  to  exhibit  some  random  behavior  independent  of 
the  signal  and  noise.  The  signal  and  noise  spectra  are 
assumed  to  be  constant  and  to  have  zero  correlation  over 
the  period  of  operation.  However,  the  method  coulO  handle 
oases  of  dependence  in  the  probability  sense  between  random 
behavior  in  the  system  and  time- variation  in  the  input 
signal  and  noise  spectra. 

Under  the  assumptions  listed  above,  the  mean  square  error 
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opeotrum  takes  the  form  of 

W‘(u)  = 5*+  (SVm')M^-  2 s’  M C=s(o<).  (18) 

The  expected  mean  square  error  spectrum  Is  therefore 

E [ e*(«)l  = S’  + (S’4  N‘)  E [M’}  - 2 5*  E f M Cos w],  ( 19 ) 

The  Closed  loop  gain  (M)  and  phase  shift  (o<)  are  assumed 
to  be  functions  of  some  system  parameter,  T*  , which 
exhibits  some  random  behavior.  The  two  Expected  Values 
are  generally  evaluated  by  Integration,  for  example: 

E[M‘(T)]=  ^ MVr)-p(r)'dT, 

where  p('T)  1®  "the  probability  density  function  of  T . 

If  the  system  environment  causes  T*  to  take  on  any  of  several 
discrete  values  rfther  than  vary  continuously,  or  If  con- 
tinuous variation  can  be  reasonably  approximated  by  dis- 
crete values  and  a discrete  probability  density  function, 
the  integration  Is  replaced  by  summation  over  all  the  random 
values  of  T. 

The  problem  of  desl<;nlng  the  optimum  system  under  these 
assumptions  may  be  resolved  into  that  of  determining  the 
best  design  value  for  the  average  of  the  random  parameter  “T  . 
The  general  conclusion  illustrated  by  this  application  of  the 
method  and  by  the  examples  to  follow  is  that  a better  system 
(having  less  mean  square  error)  will  result  from  this  use  of 
probability  theory  than  would  result  if  the  random  behavior 
of  the  parameter  were  neglected,  its  average  value  being 
considered  as  a nonrandom  parameter.  Intuitively,  the  im- 
provement can  be  seen  to  result  from  the  fact  that  the  expected 
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mean  square  error  expression  contains  system  function  aver- 
ages, such  as  E[M*]  and  , rather  than  averages 

of  the  parameter  alone. 

As  an  example,  the  servosystem  composed  of  an  integra- 
tor in  the  open  loop,  system  (1)  of  figure  2,  is  considered. 
The  unoorrelated  first  order  signal  amplitude  spectrum  and 
white  noise  used  previously  (figure  1)  are  assumed  for  this 
example  also.  The  mean  square  error  spectrum  was  formulated 
in  equation  (18)  above.  The  Expected  Value  operation  may 
be  applied  to  the  spectrum,  as  in  equation  (19)>  or  to  the 
integral  of  the  spectrum  over  all  real  frequencies.  The 
operations  of  Integration  and  Expected  Value  may  be  inter- 
changed (5b)  under  the  conditions  that  the  spectrum  is  al- 
ways less  than  some  constant  and  Is  Integrable  in  the 
Riamann  sense,  and  that  the  Expected  Value  of  the  spectrum 
is  also  Integrable  in  the  Riemann  sense.  It  is  generally 
.more  convenient  to  perform  the  integration  of  the  spectrum 
over  all  real  frequencies,  by  the  method  of  residues, 
first.  Otherwise  the  integration  associated  with  the  Expect- 
ed Value  operation  may  result  in  expressions  which  are  much 
more  difficult  to  Integrate  by  residues  or  any  other  means. 
The  mean  square  error  is  as  given  in  equation  (17)  above. 

Its  Expected  Value  is 

E[?]  = -f  f [l~  E(TtV] 

The  time  constant  is  assumed  to  exhibit  a discrete  random 
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variation  such  that  it  may  be  either  of  two  each 

with  a probability  of  1/2.  The  two  values  may  be  written 
ae  and  To  “ jT?i  where  T»  is  the  average  value  of  the 

randomly  varying  time  constant,  and  2.  f jls  the  relative 
spread,  which  will  be  considered  fixed.  The  design  problem 
is  the  determination  of  the  value  of  To  which  minimizes  the 
mean  square  error.  The  Expected  Values  are  readily  evaluate 

ea.  for  eiamyle:  J.  l_  + J. L_ 

2 T.+irt:  z r.-fT;  , 

The  expected  mean  square  error  may  be  conveniently  written 

as 

E[  e’f-n]  = ^ e^(r.t)rr.)  + e*  (r.-j>c).  (2d 

The  minimization  of  this  expression  with  respect  to  the 
average  value,  To  , of  the  time  constant  is  accomplished  by 
differentiation,  subject  to  system  stability  considerations. 
However,  a graphical  presentation  better  illustrates  the 
minimization  process.  A curve  of  the  mean  square  error  vsT 
with  E=2.  (equation  17)  is  presented  in  figure  4.  As 
equation  (21)  indicates,  the  sum  of  the  ordinates  of  that 
curve  at  T»TJ  + V'^.  and  T^T^  — jfTi  is  to  be  minimized. 
Owing  to  the  dissymmetry  of  the  curve  about  its  minimum,  the 
optimum  value  of  'Vo  will  be  shifted  ITom  the  minimum  to  some 
larger  T , the  amount  of  the  sJ7‘‘^t  depending  upon  the  value 
of  ^ and  the  shape  of  the  curve.  For  a particular  example 
with  23-2  and  0.5,  the  optimum  veluo  of  To  is  found  to  have 

shifted  from  the  minimum  of  the  curve  at  1.0  to  about  1.4. 


7a  ?l6*W  Time  Constant 
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The  most  meanirigfui  prossntation  of  the  improvement  in 
system  performance  as  a result  of  this  method  is  to  compare 
three  values  of  the  mean  square  error  for  the  integrator 
system.  For  this  example,  these  three  values,  normalized, 
are:  a)  1.000  for  an  invariant  integrator  system  with 

optimum  i , b)  1.067  for  this  system  having  the  random 
time  constant  here  assumed,  but  with  T» » 1 as  a result  of 
neglecting  that  random  behavior  in  system  design,  and 
o)  1.041  for  this  system  having  the  random  time  constant 
here  assumed,  and  using  the  method  herein  presented  to 
determine  the  optimum,  Thus,  although  the  increase 

in  system  error  due  to  randomness  is  small  (6.7^),  the 
application  of  this  method  lowered  that  Increase  by 
( 67-41 )/67  or  39^. 

In  the  Immedlatoly  previous  example,  a discrete  proba- 
bility density  function  was  assumed  to  deoorlbe  the  random 
time-variations  of  the  system  time  constant.  In  ox-der  that 
the  use  of  continuous  probability  density  functions  may  not 
be  neglected,  an  example  using  such  a density  function  is 
here  presented.  For  simplicity  of  solution,  this  example 
will  use  the  same  system  function  and  input  spectra  as  were 
used  for  the  example  just  preceding.  The  uniform  probability 
distribution  (4b)  is  ohoesn  in  this  example  to  describe  the 
time-varying,  random  behavior  of  the  system  time  constant 
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It  ie  defined  as 
per)- O ior  T<a-w, 
p(r)-Vzy  for  M-w4Ti  uiw, 
p(r;=o  for  T>u+v/, 

It  Is  aketobad  at  the  right, 

where  M seen  to  be  the 
average  value  of  'T  , and 
2wis  the  extreme  spread  of  the  random  variations  of  X . 

The  variance  ot  X ^ E[(T-uy]  , iB  wV3»  The  optimiza- 
tion problem  is  to  determine  the  optimum  value  of  u in  terms 
of  the  input  spectra  and  of  the  spread,  w.  The  expected 
mean  square  error  for  the  preceding  example  as  given  in 
equation  (20)  applies  here  also.  It  is  repeated  below  for 
reference 

E(e*]  = -f -f  {l  - E[TT?r] +7^E[tJ}, 

The  evaluation  of  the  Expected  Values  is  as  follows: 

E[+]  = £krP(r).dr  = 

E [i+rr]  = ^ in 

Jubstitutlon  of  these  valuer  in  equation  (20)  yields 

Differentiation  of  this  expression,  to  determine  the  optimum 
u,  yields 

_ -r  ^7£H)u»-2Su-f’w»(r-i)-i  -j 

Equating  this  derivative  to  zero,  noting  that  system  stabili- 
ty requires  tiiat  w < u so  that  T la  never  negative,  results 
in  the  roots  u - 

^ T(S^-T)  » 


p(T> 

<1 


6 u-w 


it 


I 

U-*  w 


-►  r 
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For  the  ease  of  no  randomness  In  , [y:  =.  o) , the  roots  are 

Obtained  for  the  Invariant  Integrator  bystem  treated  earlier. 
The  optimum  value  of  the  average  of  the  randomly  varying 
time  oonstant  Is  c|  r ^ ■ 

The  other  root  specifies  a negative  time  oonstant  and  hence 
an  unstable  system.  The  optimum  value  of  u above  Is  only 
of  value  for  E > 1 • E < I , the  average  time  oonstant 

Is  always  negative  and  sometimes  complex.  As  discussed  for 
the  Invariant  case,  when  J no  Integrator  system  can 
be  an  Improvement  over  a no-pass  system. 

As  an  Illustration  of  the  results,  consider  the  case 
of  23  “2  and  wssO.5*  The  optimum  value  of  the  time  cons- 
tant has  shU'ted  from  the  Invariant  system  value  of  1.0  to 
a value  of  1.17  for  this  time-varying  case.  The  best  means 
of  presentation  of  the  Improvement  In  system  performance 
resulting  from  the  use  of  this  method  Is  to  compare  three 
values  of  the  mean  square  error  of  the  Integrator  system. 
These  values  are,  normalized,  a)  1,000  for  an  invariant 
integrator  system  with  optimum  b)  1,016  for  the  system 

having  the  random  time  oonstant  as  described,  but  with  u = 1 
as  a result  of  neglecting  that  random  behavior  when  optimiz- 
ing the  system,  and  o)  1.011  for  the  system  having  the  random 
time  oonstant  as  described,  and  using  the  method  presented 
herein  to  determine  the  optimum  u = 1.17.  Thus,  although  the 
Increase  in  system  error  due  to  randomness  is  small  (l.l^S), 


which  check  with  those 
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the  application  of  this  method  reduced  that  increase  by 
(16-1D/16  or  ni- 

Other  evaluations  of  the  two  examples  presented,  as 
well  as  different  cases  of  this  general  type,  indicate  simi- 
lar results.  The  increase  in  system  mean  square  error  due 
to  such  random  behavior  is  not  great,  but  this  use  of  the 
Expected  Value  operation  to  determine  the  true  optimum  value 
for  the  average  of  that  randomness  recovers  typically  of  the 
order  of  magnitude  of  a third  of  the  increase. 

The  use  of  probability  concepts  was  shown  above  to  be 
necessary  for  the  optimum  design  of  those  systems  which  are 
in  essence  similar  to  the  examples  treated.  It  is  of 
interest  to  note  that  the  results  of  those  examples  are  of 
more  general  application  than  indicated  by  the  particular 
problem  stated*  As  an  illustration  of  the  generality  of 
application,  the  results  are  here  Interpreted  to  be  the  solu- 
tions of  two  different  problems.  The  first  is  to  indicate 
the  greatest  range  of  random  variations  that  can  be  tolerated 
in  the  value  of  a parameter  without  increasing  the  system 
mean  square  error  appreciably.  A possible  application  is 
the  specification  of  the  largest  thermal  coefficient  of  a 
parameter  that  could  be  tolerated.  The  second  problem  is  to 
optim: ZQ  the  average  of  the  performances  of  a largo  number  of 
systems  which  are  the  same  except  that  their  construction 
involves  one  or  more  elements  having  values  within  some 
manufacturing  tolerance.  The  proper  nominal  (average)  value 
of  the  element  and  the  tolerance  required  are  the  desired 
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results.  That  the  method  presented  herein  can  determine 
such  results  has  been  adequately  shown. 

The  oases  described  and  carried  to  completion  in  this 
part  of  the  dissertation  have  covered  only  a very  few  of 
the  possible  sets  of  conditions  on  random  time-variations 
in  system  parameters  and  input  spectra.  The  method  present- 
ed herein  is  readily  adaptable  to  much  more  complex  problems 
with  dependent  as  well  as  independent  random  variations. 
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CONCLUSIONS 

This  dissertation  has  presented  an  analytical  method 
for  the  optimum  design  of  servosystams  characterized  by 
random,  time-rarying  parameters.  The  inputs  to  the  systems 
are  described  by  signal  and  noise  voltage  spectra  which  also 
are  permitted  to  be  time-varying.  The  method  of  optimiza- 
tion is  limited  in  application  to  linear  systems,  and  it  uses 
the  criterion  of  minimum  mean  square  error.  A spectrum  of 
mean  square  error  is  derived  and  is  shown  to  be  of  a form 
which  is  different  fi*om,  although  equivalent  to,  that  common- 
ly found  In  the  literature.  The  form  derived  herein  shows 
promise  of  easier  application  to  some  practical  problems. 

Two  procedures  are  introduced  for  the  application  of  the  mean 
square  spectrum  to  the  optimization  problem.  Each  is  Illus- 
trated by  application  to  several  examples.  A formulation  of 
the  general  solution  for  the  more  realistic,  second  procedure 
is  given. 

The  method  presented  is  not  restricted  in  application 
to  servo  or  feedback  systems.  It  may  be  as  readily  applied 
to  any  linear  system  or  network,  especially  those  containing 
random,  time-varying  parameters  or-  input  spectra.  The  most 
Important  use  of  the  method  is  felt  to  lie  in  its  application 
to  problems  that  are  characterized  by  randomness  in  the 
parameters  of  the  system  or  network  involved. 
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FUTURE  INVESTIGATIONS 

The  following  suggestions  Indicate  a few  of  the 
opportunities  for  future  work  In  this  field.  An  analytical 
investigation  of  the  limitation  on  the  rate  of  random 
time-* variation,  a«  described  qualitatively  above,  would  be 
a desirable  extension  of  khe  work  contained  herein.  That 
investigation  could  lead  to  a further  extension  of  the 
method  to  cover  random  time- variations  more  rapid  than  the 
limitation  now  permits.  Another  extension  of  this  work 
would  comprise  the  solution  of  the  general  formulation 
given  above  for  some  more  general,  physically  realizable 
system  transfer  functions. 

A closely  allied  problem  Is  the  classic  one  of  the 
determination  of  the  size  of  a statistical  seunple  of  a 
random  signal  which  Is  required  to  establish  Its  spectral 
density  or  autocorrelation  function  to  within  some  speci- 
fied accuracy. 
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APPENDIX  A 
EXPERIMENTAL  CHECK 

This  appendix  contains  a description  of  an  experi- 
jnontal  oheok  on  the  use  of  the  Expected  Value  operation 
in  the  method  of  this  dissertation.  The  solutions  of  two 
examples  of  random  time-variation  in  a system  parameter 
are  obtained  analytically  and  compared  with  experimental 
results.  The  system  chosen  for  these  two  examples  is  a 
simple  lag  circuit.  Its  time  constant  is  made  to  vary 
discretely  as  was  assumed  in  certain  of  the  cases  treated 
in  the  body  of  the  dissertation.  Slow,  periodic  switching 
from  one  value  of  the  time*  constant  to  another  is  the  special 
random  time- variation  chosen,  for  ease  of  experimental 
simulation.  The  input  to  this  system  is  a single  sine  wave 
for  the  first  example  and  a random  noise  spectral  density 
for  the  second  example. 

These  two  examples  constitute  a sufficient  check  :n 
the  Expected  Value  operation.  Statistically  described 
signals  contaminated  with  noise  are  not  used  because  they 
are  not  essential  to  this  check.  In  the  paragraphs  to 
follow,  the  general  solutions  of  the  dissertation  are 
adapted  to  these  examples  and  their  solutions  are  present- 
ed. The  experiments  are  described,  and  their  results  are 
compared  with  the  analytical  solutions- 
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Single  Sine  Wave  Input 

Analytical  Solution < The  system  function  for  the 

simple  lag  circuit  may  be  vrrltten  as 

[ - 1 ^-3t«rr*(a>r) 

ujwr  “ /I  ® 

The  Input  Is  S\n((ai). 

The  steady  state  output  Is  then 


e = 


L=r  5in  tan‘W'*^))  . 


The  mean  square  output  Is  therefore 

“ 2 i+c3*r^  . 


(23) 


The  system  time  constant,  T , Is  assumed  to  taice  on  two 
values,  T<  , cuiu  Tj  , with  equal  probabilities.  Thus  the 
probability  density  funv^tlon,  P(T)  , has  the  yalues 
p (‘t'J  = ‘/z  and  p (Ta)  = ‘/a  . The  expected  mean  square 
output  Is  then 

E(  e*]  = 

E ~ i { i l+<o»-C*  2 

This  solution  assumes  that  ths  frequency  of  switching 
between  and  Ta  Is  slow  enough  that  any  transient  effects 
are  negligible.  Using  Tt  as  a base  and  setting  = k 't't  , 
the  expected  mean  square  output  la 

, i, 

£(H)-£[e]~  y(|+£o>r,*)  . (24! 


The  solution  above  may  be  conveniently  presented  as  a 
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ratio  of  the  above  result  to  that  for  no  random  variation, 
i.e.  for  K»l  and  therefore  . 

I A. 

£M  - I 1*03' V‘k‘ 

£(l)  ~ 2 (25) 

This  ratio  is  plotted  in  figure  5>  which  follows  this  ap- 
pendix, for  a particular  value  of  Caj'T',  and  as  a function 
of  K . 

Experimental  Check.  The  experimental  setup  for  this 
first  example  consists  of  the  equipment  described  below 
and  is  Illustrated  in  figure  6,  which  follows  this  appendix. 

A signal  generator  supplies  the  single  sine  wave  input  to 
the  system  which  is  an  H-C  lag  circuit.  The  system  is 
Isolated  before  and  after  by  cathode  follower  circuits, 
which  are  formed  on  the  Operational  Manifold  (George  A. 
Fhllbrlok  Researches,  Inc.).  The  capacitance  of  the  cir- 
cuit is  varied  discretely  by  adding  and  removing  addltl*  nal 
capacitance,  using  a motor  driven  switch.  The  motor,  the 
gear  train  drive,  and  the  cam-operated  micro -switch  are 
conveniently  assembled  using  the  Belocic  Instrument  Corpor- 
ation Servo  Components  available.  The  output  of  the  syst.oia, 
fed  through  a cathode  follower,  is  coupled  to  the  heater 
coil  of  a Z2J  Western  Electric  thermocouple.  The  resulting 
squared  and  averaged  output  voltage  is  read  on  a light  bear 
type  galvanometer. 

The  system  was  ilrst  checked  as  an  invariant  system 
against  the  known  attenuation  versus  frequency  characteristic 
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of  a simple  lag  oirouit  and  was  found  to  be  in  good  agree- 
ment. The  experimental  procedure  oonsiated  easentlalj.y  of 
selecting  a suitable  base  value  for  u?Ti  (this  was  chosen 
for  lOOcps,  R~  lOK,  and  0=  0.4  mfd.),  varying  the  value 
of  the  other  capacitor,  and  recording  the  galvanometer 
deflections  for  three  situations.  These  are  each  of  the 
two  positions  of  the  switch  and  the  slowly  switching 
condition.  The  experimental  ratios  correspond} ng  to  equa- 
tion (25)  are  plotted  on  the  analytical  curve  of  that 
ratio,  figure  5t  SlB  circles.  They  are  seen  to  lie  within 
2.  ^ of  the  absLytloal  solution. 

Spectral  Density  Input 

Analytical  Solution.  The  system  function  is  again 
that  of  the  simple  lag  circuit.  ^ 1 

The  input  spectral  density  chosen  is  that  expected  from 
the  General  Radio  Random  Noise  Generator,  Type  1390-A.  The 
amplitude  spectrum  may  be  written  as  _ 

where  is  the  rms  amplitude  at  high  frequency  and  ^ is 
the  lower  cut-off  ( -3db)  frenuenoy.  The  mean  square  out- 
put spectrum,  adapted  from  equation  (7),  i& 
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The  mean  square  output  Is  the  Integral  of  that  spootrum. 


JL 


(26) 


The  expected  mean  square  output,  obtained  in  the  same 
manner  as  in  the  preoeding  example,  is  thus 


E[e*]®  2 E! [ y /2T)  ] “ *2  i Ta( l+f 


Setting  Xi~k%  gives  the  simpler  form  of  the  result 
A ratio  Is  formed  as  in  the  single  sine  wave  case  above# 


ecn  "I 


(27) 


This  ratio  la  shown  as  a function  of  K In  figure  7,  for  a 
particular  value  of  Tf  and  of  j> 

Experimental  Check.  The  experimental  setup  for  this 
example  differs  from  that  of  the  preceding  example  In  that 
the  Random  Noise  Generator  Is  substituted  for  the  sine 
wave  generator,  and  an  amplifier  and  another  cathode  fol- 
lower (both  formed  on  the  Operational  Manifold)  are  in- 
serted before  the  thermocouple  heater  to  compensate  for 
the  low  output  level  available  from  the  Random  Noise 
Generator.  This  setup  l6  also  shown  in  figure  6. 

The  procedure  followed  here  is  the  eama  as  that  of  the 
preoeding  €^xample.  The  value  of  ^ is  determined  for  the 
analytical  curve  by  using  the  galvanometer  deflections  for 
two  values  of  TT  with  the  svstem  invariant.  The  ratio  of 
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the  deflections  and  the  two  values  of  X are  substituted 
In  the  analytical  eTpresslon  for  the  ratio.  It  Is 

JlisL  = -Siii.fi  ty.  A D 

T*TrO  TO+fE)  ^ • 

Therefore , 


p - '5  -R'a 

“ ■ Rr,*-r,* 

The  experimental  ratios  which  correspond  to  equation  (27) 
are  calculated  from  the  galvanometer  deflections  for  the 
base  time  constant  (T|)  alone  and  for  the  slowly  switching 
condition  of  the  system  time  constant.  These  experimental 
ratios  are  plotted  as  circles  on  the  anal]rbloal  curve  of 
that  ratio,  figure  7<  The  experimental  values  are  seen  to 
lie  within  1.5^  of  the  analytical  curve*  The  Random  Noise 
Generator  does  not  quite  have  a true  first  order  character- 
Istie  at  low  frequencies.  A first  order  filter  is  coupled 
on  the  output  of  the  generator  to  correct  this  condition. 

These  two  experiments  Indicate  a good  agreement  bet- 
ween the  experimental  results  and  the  analytical  solutions 
using  the  Expected  Value  operation.  It  is  concluded  that 
the  use  ox  this  probability  theory  operation  is  certainly 
valid,  with  one  limitation.  The  rate  of  the  random  varia- 
tion must  be  slow  enough  that  any  resulting  trcuisients 
will  have  negligible  effects  on  the  mean  square  value  of 
the  output. 


i ‘ Figure  5 

i Comparison  of 

I |j  Experimental  Points  and  Analytical  Curve 

I ji  Mean  Square  Error  Ratio  ; &(k)/£o) 

I versus  the  Random  Variation  Factor;  l( 

[ 1 1 

I Single  Sine  Wave  Input 


I 


Experimental  Setups  for  a Check  on  tke  Expected  Value  Operatint 


Comparison  of 

Experimental  Points  and  Analytical  Curve 

Mean  Square  Error  Ratio  : £ (k)/£(i) 
versus  the  Random  Variation  Factor  k 

Spectral  Density  Input 

i.o]  \ 

Experimental 
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^\PPMDIX  B 

OPTIMIZATION  OF  A FIRST  ORDER  SYSTEM 


This  appendix  desoribes  the  optimization  procedure  for 
system  (3)  as  shown  In  figure  2.  The  results  of  this  opti- 
mization were  aisoussed  in  the  body  of  this  dissertation 
and  were  presented  in  figure  3.  The  open  loop  of  this 
system  consisted  of  a simple  lag  plus  an  ideal  amplifier. 
The  closed  loop  system  treuisfer  function  is 


b 

K+ 1 + 


) 


K 

yfK+i)^+  w*r»' 


The  signal  euid  noise  are  assumed  to  be  unoorrelated.  They 
were  shown  in  figure  ].  and  are 

SCu)  - ft  , N(w5  » rj . 

The  signal  to  noise  ratio  is  as  defined  before:  . 

The  proper  form  of  the  mean  square  error  spectrum  is 
equation  (16),  repeated  here  for  reference. 


p(w)=  5‘+  (S‘+N*)M*-  25*MCos(«»). 


(16) 


Substitution  of  the  assumed  system  function  and  input 
spectra  yields 

(l3*+F (k-h)*+  i*^*'r* 

/3\  KCKH) 

U>^+S^  rkf  0*  4- . 


2 
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(2B) 


The  mean  square  error  is  the  Integral  of  this  speotram  over 
all  real,  positive  frequencies.  By  the  method  of  residues, 

f l + (Ki-i)ST  ^ ] 

2 1 5 (K*i)(K*hsr)  r(K+i)  J, 

It  may  *)e  written  In  terma  of  the  eignal  to  noise  ratio » 23  , 
and  a dimensionlesa  time  oonatant»  as 

•s*  - JC  fw  l + (Kti)T_  , K!_  \ 

® ~ * SE*  (k+i)(k+-i+t)  T(mO  /, 

It  l8  this  mean  square  error  that  is  to  be  sdnimized  by 
proper  choice  of  the  system  time  constant, 7T,  and  the 
gain,  Ka  Mnimlzatlon  requires  tbat  the  partial  drlvatives 
of  €*  with  respect  to  T and  K be  equal  to  zero.  Setting 
the  first  partial  equal  to  zero  yields 

ISa  ■n:  /3^.  K(Kt2) 

Tx  z fK+l+T)*- 

~r  - i'k4-i'^U' ± E7K(K+2)' 

' ~ fEyi<7K+a)'-K]  [EVK(Kta)>KJ  . 


The  value  of  T using  the  negative  sign  above  yields 

which  is  always  negative  (for  K>0)  and  there  results  an  un- 
stable system.  The  root  using  the  positive  sign  is 

I 

(29> 


t; = K (K+o 


E</k7k+2T-K 


Inspection  of  this  root  by  substitution  in  the  mean  square 
error,  equation  (28),  and  by  other  means  establishes  it  as 
the  optimum  time  constant.  As  a check,  this  root  reduces 
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to  the  optimum  time  oonstant  for  the  simple  lag  system 
(system  (2)  of  figure  2)  when  K is  set  equal  to  unity. 

The  optimum  time  constant,  equation  (29)  Is  substituted 
into  the  mean  square  error,  equation  (28),  and  the  partial 
deriyatlre  with  respect  to  K is  equated  to  zero.  There 
results  a cubic  in  K , 

( l-E*)  K’+  (2-'/r)K’+  (£’‘-5E*)K  + ZZ*  = 0, 

which  may  be  factored, 

[0-Z*)K-2E']-f  K*  + 2K"E*J  = O, 

to  determine  the  three  roots.  These  are 

K,  = -l-^/2^  , Ka=-l+>/2^,  Kj=2Zya-r,>). 

In  general,  these  roots  Indicate  the  maximum,  minimum,  and 
turning  points  of  the  function.  Rather  than  form  the  second 
partial  of  s'  with  respect  to  K to  check  these  roots  for 
a minimum  of  15^  , they  are  substituted  In  e*  and  the  results 
are  inspected.  Substitution  of  the  first  root  yields  a 
negative  value  of  ^ , an  Impossible  physical  situation 
resulting  from  K being  sufficiently  negative  and  hence  the 
system  being  unstable.  The  third  root  yields  a mean  square 
error  which  is  always  the  sumo  as  that  for  a no-paos  aystesm. 
Substitution  of  the  second  root  results  in  a value  of  6® 
which  Is  always  less  than  that  for  a no-pass  system.  It  Is 
this  second  root  that  Is  the  optimum  gain  within  the 
limitations  of  system  stability.  The  optimum  time  oo.nstant 
is  thus  equation  (29)  evaluated  for  Kst- 

(-|Vn/E^) _ A 

C /(VcHr + (/^+r  - ir  J 
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The  optimum  system  has  a minimized  mean  square  error  of 

e*  = f 2 (- 1 + v/E^)  , , 

Z 6E*  > ' (30) 

whldh  results  .rom  substitutii:^  the  optimum  gain^ 

K=  -1+\/e^T  , aM  the  optimum  time  constant,  ^/S  , 
In  equation  (28).  It  is  this  mean  square  error,  equation 
(30),  which  Is  plotted  In  figure  3 above  and  Is  there  dis- 
cussed. 

The  surface  generated  by  ^ as  a function  of  H and 
T iB  of  great  value  In  visualizing  this  optimization 
process.  For  that  reason,  a contour  map  of  that  surface 
(for  s 2.)  Is  Included  here  as  figure  8. 


Contour  Map  of  the 
Mean  Square  drror  Surface 
for  System  3 

Normalized  to  the  mean 
sau^rt  error  of  a n^^pass  system 
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APPINDIX  0 

OPTIMIZATION  OP  A SECOND  ORDER  SYSTEM 
This  appena?.?:  deaoribes  the  optimization  of  tbe 
second  order  fijrstem  described  as  the  fourth  Invariant 
system  in  the  body  of  the  dissertation.  It  has  a closed 
loop  transfer  function  which  Is  the  product  of  two  simple 
lags  with  time  constants  T<  and  Tt  . It  is  written  as 

(i+joj-trAi +ja?-G)  , 

The  signal  and  noise  inputs  are  asaumad  to  be  uncorrelated 
and  are  described  by  their  amplitude  spectra  as  shown  in 
figure  2 above  and  as  written  below. 

5cw)=  /3  = 

The  mean  s<iuare  error  spectrum  for  these  assumptions  is 
equation  repeated  here  for  convenience. 

"Piu)=  S*+  ( S*+N*)M*-  2 S*M  Cosccc), 

Substitution  of  the  system  gain  ^nd  phase  shift  and  the 
signal  and  noise  amplitude  spectra  yields 

! 

3 l-CO»-t7r. 

Cl>“+  f » ' . 

The  mean  square  error  is  formed  by  integrating  this  spectrum 
over  all  real,  positive  frequencies  by  tne  method  of  real- 


/3^ 


*^n 


■) 
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dues. 


® -a\T  >. 


Ttils  zray  be  written  In  terms  of  the  signal  to  noise  ratio, 
l^=/9/^*l  , and  two  dimensionless  time  constants,  t 
and  Ta  * as 


® ■ 2 75? 


(T+T.Kl+'nXl+Ta)  I, 


(31) 


A^uatlng  to  zero  the  partial  of  with  respect  to  Ti 

yi«ws  i-r,  (g-o  ± s:j(2t.-i)L%‘cs'*i)-\T 


For  the  special  case  of  Ta  ^O,  these  roots  check  with 
those  derived  for  the  Integrator  system  (system  (1)  of 
figure  3)»  Owing  to  the  complexity  of  the  roots,  a 
graphical- analytical  approach  Is  used.  Specific  values 
of  Zj  are  substituted  in  equations  (32).  Those  equations 
are  plotted  In  the  T versus  Ts  plane  and  give  the  curves 
along  which  the  partial  derivative  of  8 with  respect 
to  T|  Is  zero.  The  similar  curves  along  which  the  partial 
of  ^ with  respect  to  Ti  is  ^ero  are  also  plotted.  These 
latter  curves  are  obtained  by  Interchanging  T and  Ti  In 
equation  (3<i)f  making  use  of  the  symmetry  of  8*  with  respect 
to  Ti  end  Tt . The  Intersections  of  these  curves  are 
expected  to  Indicate  the  conditions  for  minimum  aiid  maxi- 
mim  and  possibly  a saddle  point  of  the  surface  of  as 

a function  of  Ti  and  Ta,  . Only  the  intersections  in  the 
first  quadrant  are  of  interest  as  that  is  the  region  of 
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positive  time  oonatante  and  henoe  system  stability.  Evalua» 
tlon  of  & at  those  inter seotlona  and  at  adjacent  points 
reveals  that  the  Interoeotions  in  the  first  quadrant  indicate 
maxima  and  saddle  points  of  ^ . The  minimum  values  of 
are  to  be  found  at  intersections  in  the  other  quadrants,  out- 
side the  region  of  system  stability.  Therefore  the  restrict- 
ed minimum  of  ? must  lie  on  the  boundaries  of  the  first 
quadrant,  i.e.  or  T*»0.  These  boundaries  correspond 

to  the  previously  treated  first  order  system  having  an 
integrator  in  the  open  loop.  It  was  optimized  in  the  body 
of  the  dissertation.  The  optimum  second  order  system,  for 
the  assumptions  of  this  case,  is  therefore  a first  order 
system.  Optimization  dictates  that  one  time  constant  be 
zero  and  the  other  be  us  given  by  equation  (33)  with 
equal  to  zero  and  the  positive  sign  used.  That  value  is 

T = 

Consideration  of  the  shape  of  the  surface  generated  by  ^ 
as  a function  of  X and  XL  of  groat  value  in  visualizing 
the  situation.  A contour  map  of  th;'t  surface,  for  3C-2  , 
is  Included  here  as  figure  9«  Investigation  of  the  boundary 
between  mean  square  error  less  than  and  greater  than  that 
for  a no-pass  system  generally  leads  to  a simpler  polynomial 
than  the  general  contour  of  the  surface.  It  is  also  of 
value  in  visualizing  the  situation.  Another  use  of  the 
surface  representation  is  that  it  presents  the  solutions 
of  the  restricted  problem  of  one  time  constant  being  fixed 


Contour  Map  of  the 
Mean  Square  Error  Surface 
for  the  Second  Order  System 


Noirmalizeci  to  the  wean 
sejuare  of  a jid-pass  system 

E»2 
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In  value.  The  optimum  value  of  the  remaining  time  constant 
is  readily  discovered  by  inspection  of  the  proper  section 
through  that  surface. 


